feat(extract): model Go native channels as gochan Channel topology - #1949
feat(extract): model Go native channels as gochan Channel topology#1949ilyabrykau-orca wants to merge 1 commit into
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
|
CI retrigger (no-op amend): |
f962097 to
e41c7df
Compare
|
Field census landed (same-day baseline vs this branch, repo @ f555e5ce):
Against the issue's source counts (~210 sends, ~500 receives): 162 sends and 256 receives captured — the gap is exactly the deferred shapes (locals that never escape, |
DeusData
added
enhancement
|
Thank you for documenting the measured Go channel gap, the extraction-only boundary, and the positive and negative tests. Because this draft changes graph identity and cross-file channel topology, we need more time to review the design carefully. Please keep it in draft for now; the contribution queue is quite full, but we will come back with a grounded decision as soon as capacity allows. |
Go's own concurrency primitives were invisible: the Go channel extractor only classified gorilla/nhooyr WebSocket send/receive, so a channel-plumbed event pipeline (250 make(chan ...), ~710 send/receive sites on the measured repo) produced zero Channel nodes and trace_path stopped dead at every send site. v1, extraction-only - the existing per-file materializer (create_channel_edges_for_file and its parallel twin) already builds Channel nodes and EMITS/LISTENS_ON edges from CBMChannel records: - send_statement (x <- v) -> EMIT, unary <- -> LISTEN. Both are channel operations BY GRAMMAR, so no type inference is needed for precision - unlike the WebSocket name heuristics. select comm clauses are covered for free (they contain the same node kinds). - Channel identity: the package-qualified tail identifier (module_qn + '.' + field/var name), transport "gochan" - distinct from "websocket", whose classifier is untouched. Same-package cross-file producer/consumer pairs join on one node. - Deliberately deferred (documented in DeusData#1930): element types on the node, go statements (CROSS_ASYNC), and for-range receives - range needs the operand's TYPE to know it is a channel, and a name-shape guess would be the DeusData#1932 anti-pattern. Also fixes a latent gap this exposed: enclosing_function_qn returned a BARE name, which never matches any def QN, so every channel edge (the WebSocket ones included) silently degraded to the file node through find_channel_source's fallback. It now returns module_qn.name, with the file-node fallback preserved for shapes it cannot express. Reproduce-first (RED with extract_channels.c stashed): extract_go_native_channels (EMIT+LISTEN records, package-qualified names, unary minus not mistaken for a receive) and pipeline_go_native_channel_topology (one gochan Channel node, EMITS from Produce and LISTENS_ON from Drain across files). 643 green across extraction/pipeline/registry. Part of DeusData#1930 Signed-off-by: Ilya Brykau <ilya.brykau@orca.security>
|
Rebased onto current main (staying in draft as asked). Conflicts were test insertion-anchor drift in tests/test_extraction.c and tests/test_pipeline.c — re-anchored; all source auto-merged. Full scripts/test.sh green on this head. |
e41c7df to
91a3c6a
Compare
2 participants
Part of #1930 — draft implementation-proposal per the extraction points sketched in the issue body; will mark ready on a maintainer design ack.
What
Go's own concurrency primitives were invisible: the Go channel extractor only classified gorilla/nhooyr WebSocket send/receive, so a channel-plumbed event pipeline (250
make(chan …), ~710 send/receive sites on the measured repo) produced zeroChannelnodes andtrace_pathstopped dead at every send site.How (v1, extraction-only — the existing materializer does the rest)
create_channel_edges_for_fileand its parallel twin already buildChannelnodes +EMITS/LISTENS_ONedges fromCBMChannelrecords, so the whole change lives in the Go extractor:x <- v(send_statement) →EMITS;<-x(unary) →LISTENS_ON. Both are channel operations by grammar — no type inference needed for precision, unlike the WebSocket name heuristics.selectcomm clauses are covered for free (same node kinds).module_qn.field-or-var), transportgochan— distinct fromwebsocket, whose classifier is untouched. Same-package cross-file producer/consumer pairs join on one node.gostatements (CROSS_ASYNC), andfor range chreceives — range needs the operand's type to know it's a channel, and a name-shape guess would be the exact anti-pattern Meta: Go / cgo / Go+C extraction — tracking issue #1932 catalogues.Also fixes a latent gap this exposed:
enclosing_function_qnreturned a bare name, which never matches any def QN — so every channel edge (the WebSocket ones included) silently degraded to the file node viafind_channel_source's fallback. It now returnsmodule_qn.name, fallback preserved.Tests (reproduce-first — RED with
extract_channels.cstashed)extract_go_native_channels: EMIT+LISTEN records, package-qualified names, unary minus is not a receive.pipeline_go_native_channel_topology: onegochanChannel node with"transport":"gochan",EMITSfromProduceandLISTENS_ONfromDrainacross files (RED:cc == 0, expected 1).scripts/test.shvenue leg green; clang-format clean.Field census
Landed (same-day baseline, repo @ f555e5ce): Channel nodes 2 → 205 (203
gochan); EMITS 1 → 162; LISTENS_ON 2 → 256; 252 distinct source functions. The gap to the raw source counts is exactly the deferred shapes (non-escaping locals,for range ch,gostatements). Full table in the census comment below.#1932 tracks the family. Related: #1114.